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CHAPTER I 



INTRODUCTION 

The last decade has brought about many changes in the field of 
vocationaJL education. Shifts of emphasis have occurred and new instruc- 
tional sireas have emerged. The advent of Public Law 88-210, the Voca- 
tional Education Act of 1 ^ 63 } has furnished the field with the legisla- 
tive and monetary backing needed for e3q>loring new areas of instruction 
and for developing new curricula teaching materials and methods of pre- 
sentation and dissemination. 

The vocational agriculture program, in keeping pace with a rapidly 
changing society, has become a complex and multi- dimensional educational 
endeavor involving instruction not only for those who intend to farm, 
but also for those seeking employment in off-farm agricultural occupa- 
tions. E3!5)ansion has occurred in such a way that in addition to provid- 
ing instruction in farm production and management, courses have been 
added in specialty areas such as ornamental horticulture, agricultural 
mechanization, conservation, fo. estry, and agricultural business. The 
development of entry level skill is the prime objective of these new 
instrUwCional programs just as entry into farming has always been the 
prime objective of the traditional vocational agriculture program. 

This general e:q)ansion in the vocational agriculture program has 
resulted in three shortages: teachers trained in specialty areas; 
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instructional materials relating to these new areas; and media through 
which these instructional materials may be presented. 

This study is an experiment employing one of these new instruc- 
tional media. It is designed to detejrmine the relative effectiveness 
of supplementing programed instruction wn.th blocked versus spaced review. 



Stat eme nt of the Pro blem 

Since B. i*’. Skinne’?’s IS^h a.rticle, "The Science of Learning and 
the Art of Teaching, " introducing the concept of machine teaching, appeared 
in Ha rvard Edu c atio n al.. Pev3. ew, the professional literatiire in education 



has been literally flooded with artic3.es and research reports on programed 
instruction. Prcgrems and program holding devices have also been forth- 
coming at a rapid rate. Programs. *63^ lists 352 programs available in 
ten subject matter areas. This is almost a 300^ increase over the 122 
program.s reported available the previous year. 



It is evident at this point that programed instruction is here 
to stay . Programs are not only becoming more prevalent in the pbulic 
school, but industry is also ea^loying programed instruction at a rapid 
rate. For exaniple, Du Pont recent3.y offered, for purchase by the public, 
ninety programs in industrial subjects. Another factor, the commercial- 
ization of programed instruction by publishing companies, and the sales- 
manship of their representatives, will also contribute to the increased 
use of programs. 



1. U. S. Department of Health, Education and Welfare, Programs, *63; A 

“^9 ■ Instruc tion (Washington; Government Printing 
Office, 19^3), p. vi. 

2. "Du Pont Offers Program.ed Instruction Courses," Du Pont Agricxatural 

33, no. 1 (Spring I966), p. 3. 
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Programed instruction has been proven to be an effective method 
of teaching. Numerous studies have shown that persons learning from pro- 
grams perform as well^ or better, than persons learning from conventional 
iaethods of teaching. "It seems clear that teaching machines, like tele- 
vision, books, or a lecturer, can teach. Repeated studies have shown no 
significant differences in learning by machine cts con^ai'ed to other more 
conventional modes." As programers develop more skill in their writing, 
and as improved ways of using programs are discovered, these devices 
should become even more effective. 

Research in the area of programed instruction has been primarily 
concerned with comparing programed instruction with other teaching methods 
and with manipulating variables internal to the programs themselves. A 
definite void exists in that very little research has been conducted on 
in5)roving the use of existing programs in the classroom. "A common 
question asked revolves about the role of teaching machines in the total 
instructional situation. Are they designed to replace the normal instruc- 
tional pattern, or are they just another aid?"^ "Our knowledge of how 
to best utilize them (programs) is meager and imperfect."^ These are 
examples of statements found in the literature when one seeks to answer 
the question of how programs should be used in the classroom. 

One recommendation frequently made as to how teachers should incor- 
porate programs into their classroom activities is, "Teachers should 



3. Egan G, Guba, "Teaching Machines Are Here to Stay," Theory Into 

^actice , vol. 1, no. 1 (February 1962), p. 6. 

4. Desmond L. Cook, "The Automization of Socrates," Theory Into Practice, 

vol. 1, no. 1 (February 1962), p. IJL. 

5. Guba, p. 6. 
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develop, independently of any program, a set of objectives to be attained 
by students at the end of a particular subject."^ This statement led the 
vriter to take the following stand with regard to incorporating programed 
instruction into the overall instructional program in the classroom: If 

the educational objectives are clearly defined for a particular subject, 
then the objectives posed for the program (program objectives) may be 
studied to see which of the educational objectives could be accomplished 
by ac 3 ministering the program. To the extent the program accomplishes the 
educational objectives, the program may be used as the sole means of in- 
struction; to the extent the program does not accomplish the educational 
objectives, the program must be either rejected or supplemented. 

Three situations emerge from this stand: 

Situation 1 --The program alone, accomplishes all of the educa- 
tional objectives posed for a particular subject. 

Situation 2 - -The program, alone, accomplishes all of its stated 
objectives, but only some of the educational objectives posed for a par- 
ticular subject. 

Situation 3--The program, alone, only partially accomplishes 
the objectives for which it was designed. 

The third situation is the one with which this e3q>eriment is con- 
cerned. Whatever the cause of the failure of a program to teach a student, 
whether the program was not well- written, whether the students were poorly 
motivated, whether the teacher expected a higher level of achievement than 
did the program writer, etc., the teacher is somewhat limited in what he 



6. Richard W. Burns and Mary B. Craik, "Using Programs in the Classroom," 
Audiovisual Instaruction. vol. 10 , no. 9 (November I965), p. 697. 
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can do to correct the situation. He can either discontinue use of the 
program, or he can supplement it with additional teaching.*^ 

In this situation a common form of supplementation is review, 
so the teacher asks himself, "Should I or should I not conduct a review 
In such a case, the question of when the review is to be conducted must 
be answered. That one should conduct a review and that the review should 
be spaced rather than blocked will be hypothesized later. 

Review of Related Research 

A search for related studies was conducted in Education Index > 
by the H. W. VJilson Company; Research Studies in Education , by Phi Delta 
Kappa, Inc.; and The Research on Programed Instruction; An Annotated 
Bibliography , by Wilbur Schramm. Although the body of research done on 
programed instruction is quite voluminous in size, very little of it can 
be related to the current experiment because of the nature of the variables 
treated in this esqperiment , namely blocked review versus spaced review as 
a supplement to programed instruction. The search yielded only one study 
which could be related directly to the current experiment. 

Reynolds and others conducted a series of experiments to study 
the effects of repetition and spaced review on learning and retention 
from programed instruction. In the introduction of their final report, 
which is dated December 1964, they make a statement which lends support 
to the writer in his inability to locate further related studies: "No 

attempt has been made as yet to determine the effects of repetition and 

7* It is important for the reader to understand that this study will 

not deal with teacher- written programs, but with published programs. 
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spaced review in programed instructional materials." 

Spaced review was defined by Reynolds as a sequence of learning 
conditions in which a second topic is interpolated between successive 
presentations of the first topic. In his experiments only one topic was 
reviewed and this review was facilitated by interpolating additional 
frames of programed instruction, on the topic to be reviewed, between 
following topics in the program. In other words, only one topic was 
reviewed and it was reviewed at intervals determined by the length of 
the following topics. 

Although Reynolds' concept of spaced review is slightly different 
from that he3.d by the writer, there is enough similarity to make his find- 
ings relevant to the current experiment. Reynolds found that "... spaced 
review alone rather than repetition and review in combination was found to 
be the variable facilitating learning and retention."^ He further stated 
that ". . . spaced review produced significant increases in learning 
which persisted and even increased, through a 3- week retention interval. 

Reynolds was somewhat reluctant to relate his findings to those 
of learning theorists and experimental psychologists whose work took 
place primarily in laboratory settings. He stated, "The consistent and 
significant effects of spaced review (as found in his experiments) are 
difficult to relate to previous laboratory research because the manner 
in which review was distributed in the present studies does not replicate 

8. James H. Reynolds, et al .. Repetition and Spaced Review in Prograj^fip*^ 
Instruction (Ohio: Aerospace Medical Research Laboratories, Wright- 

Patterson Air Force Base, 1964), p. 1. 




9» Ibid ., p. 26. 
10. Ibid . , p. 29. 
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in several respects either the distribution of practice or the retroaction 

11 

paradigms used in the laboratory." 

Studies of the type mentioned by Reynolds involving primarily ex- 
perimentation in laboratory settings are the ones from which the major 
portion of the theoretical framework for the current study were derived. 

Summaries of the general findings of these studies are presented 

T P 

in "Ten Years of Massed Practice on Distributed Practice" by Underwood, 

and in a review of the research on retroaction inhibition by Slamecka and 
13 

Cerasco. The general conclusion regarding review drawn in these two 

articles was summarized very well by Sohn: "Learning theorists have 

pointed out and psychologists have demonstrated experimentally that 

ttl4 

periodic review facilities learning and retention. From this conclu- 
sion he hypothesized that "Periodic review during a program helps the 
students to organize the small bits into a sensible structure." This 
hypothesis of Sohn’ s received careful consideration by the writer at the 
time the two hypotheses for the current study were posed. 

The current experiment differed from Reynolds* experiments to 
approximately the same degree that his differed from previous laboratory 



11. James H. Reynolds and Robert Glaser, R epetition and Spaced Review in 

the Learning of Connecte d Discour se "(Washington: U. S. Office of 

Education, ‘'’l9S3)7”Project ~NcT. 13^37 P» ^0* 

12. B. Jo Underwood, "Ten Years of Massed Practice on Distributed .Practice," 

Psychological Re view, vol. 68 (1961), pp. 229-24?. 

13. No Jo Slamecka and J. Cerasco, "Retroactive and Proactive Inhibition 

of Verbal Learning," Psychological B ulletin , vol. 5? (i960), pp. 49- 

60. 

14. David A. Sohn, "Programed Instruction: A View from the Classroom," 

Programs, Te ach ers, and Machines. Alfred de Grazia and David A. 

Sohn, ed. (New York: Bantam Books, 1964), p. I83. 
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research in that the way in which review was spaced in the current exper- 
iment did not replicate the way in which review was spaced by Reynolds. 

It further differed in that it compared spaced review with blocked review, 
it employed a filmstrip in the review procedure, and in the case of the 
treatment group receiving spaced review, the review immediately followed 
the programed topic. It was similar to Reynolds* study in that review 
was compared with non- review. 



Theoretical Framework 

The theory for this study incoiporated the principles of frequency 



and whole versus part learning which together gave rise to the following 
postulates: 



1. If there is a constant context within units of the 
learning task and a disparate context between units of the 
learning task, learning can be increased by any method that 
separates the learning task into its separate units. 

2. In the case of a lengthy and/or complex learning task, 
review between units of the task will result in more learning 
than review only at the end of the entire learning task. 

3. The more frequent a learning activity is ejcperienced, 
the greater is the amount of the resultant learning. 

If the assumption is made that there is a constant context within 



units of the learning task and a disparate context between units of the 
learning task, then it follows from postulate 1 that a structured review 
session would be a method of separating the learning task into its sepa- 
rate units. By combining postulates 1 and 2 one could deduce that it 
would be best to present the review between units of the learning task 
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rather than to have blocked review at the end of the task. From postulate 
3 it follows that review of the learning task vrould increase the frequency 
of the learning e:xperience so that one could say that a group having review 
should do better than a group having no review. 

Four general deductions may be drawn from the theory and these 
axe stated as the hypotheses to be tested: 

Hypothesis 1: Students who have program texts supplemented with 

review will learn more than those who have only programed texts, as 
mv^asured by a test for learning administered immediately after the 
instructional period. 

Hypothesis 2: Students who have programed texts supplemented 

with spaced review will learn more than those who have programed texts 
supplemented with blocked review, as measured bjr a test for learning 
administered immediately after the instructional period. 

Hypothesis 3i Students who have programed texts supplemented 
with review will retain more than those who have only programed texts, 
as measured by a test for retention administered after an interval of 
30 days. 

I^ypothesis 4: Students who have programed texts supplemented 

with spaced review will retain more than those who have programed texts 
supplemented with blocked review, as measured by a test for retention 
administered after an interval of 30 days. 

Importance of the Study 

The findings of this experiment will add to the body of knowledge 
on the use of review as a supplement to programed instruction. Campbell 
and Stanley speak of experimentation as: (l) the only means for settling 
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disputes regarding educational practice, (2) the only way of verifying 
educational improvements, and (3) the only way of establishing a cumula" 
tive tradition in which improvements can be introduced. If this state- 
ment is tahen as the role of ej^perimentation, the proposed 63{periment 
conpliments this role in that it will; (l) provide evidence regarding 
the value of supplementing programs with review, (2) provide data which 
will demonstrate the superiority of one method of review over another, 
and (3) add to the body of knowledge in the field in a cumulative tra- 
dition by either supporting the theory from which the study is drawn or 
by supplying evidence contrary to the theory. 

The purpose of this study was to test the theory presented on 
page eight, using programed instruction. It was desirous to krow whether 
the postulates inherent in the theory are generalizable to programed 
learning tasks. The outcomes of this experiment will supply empirical 
evidence on this. 

This stated, pinpose parallels the goal of educational research 
workers as expressed by Travers, ". . .to produce a body of knowledge 
consisting of generalizations about behavior which can be used to pre- 
dict behavior in educational situations and for planning educational 
procedures and practices.' 

Design of the Experiment 

The basic model from which the design for this experiment was 



15. Donald T. Campbell and Julian C. Stanley, "Experimental and Quasi- 

Experimental Designs for Research of Teaching, " Handbook of Research 
in Teaching , N. L. Gage, ed., 1963, p. 172. ■ ^ 

16. Robert M. W. Travers, An Introduction to Educationa-1 Reseoi^f*^ (New 

York: Macmillan Co., 1964 ^ p. 6. — 
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constructed is defined by Campbell and Stanley as The Posttest - Only 

Control Group Design , and is classified, according to them, as a true 

17 

experimental design. The basic elements of this design are: (l) ran- 

domization, (2) treatment, and (3) posttest. This is presented diagram- 
matically in Figure I, below. 



Figure I . Paradigm of the Posttest - Only 
Control Group Design. 



Randomization 





Treatment Group 


Treatment 


Post test 





I 


Yes 


Yes 


> 


Control 


None 


Yes 



In building upon the above design to construct the design for the 
current experiment, three elements were added: (l) a third treatment 

group, (2) a measure on a covariable, and (3) a re-test for retention. 

A diagram of the design for the current experiment is shown in Figure II. 
The three treatment groups were: (l) programed instruction plus blocked 

review, (2) programed instruction plus spaced review, and (3) programed 
instruction only. The dependent variables were student scores on a test 
for learning and on a re- test for retention given thirty days later. Indi- 
vidual student scores on the Nelson-Denny Reading Test provided a measure 
of the covariable to be used in the analysis of covarieince, the statistical 
analysis chosen for the experiment. 



17. Campbell and Stanley, p. 195* 

18. M. J. Nelson and E. C, Denny, The Nelson-Denny Reading Test , Revised 

ed.. Form A (Boston: Houghton Mifflin Co,, I960}, 



Figure II . The Experimental Desig: 
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CHAPTER II 



METHOD 

The experiment was conducted in a series of rather distinct steps. 
Step 1 was undertaicen immediately after the problem was identified and 
Step 10 terminated the experiment. 

Step 1: Selecting the Program 

In selecting an appropriate programed unit for use in the experi- 
ment, four criteria were kept in mind: (l) the program must provide 

instruction on a subject normally taught in vocational agriculture; 

( 2 ) the program must be of sufficient length so that differences in 
treatment can be observed with some assurance; ( 3 ) the program must be 
of high quality; and, (4) the subject matter must be difficult enough 
to be challenging for 9th grade students. With the help of a bibliography 
of the programed instructional material available for use in secondary 
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schools. Parliamentary Procedure: A Programed Introduction , by John W. 

Gray and Richard G. Rea, was selected for use in this experiment. This 
program, which was published in I 963 by Scott, Foresman and Company, 
meets the above listed criteria. It is in textbook form, linear in 
format and, requires from five to seven hours to complete. 



19 . The Center for Programed Instruction, Programed Instruction Materials 
1964- * 65 (Teachers College, Columbia University, 1965). —— 
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step 2t Determi nin g Which Review Proc edure to Use 
In selecting which review procediire to use in the e3q>eriment 
several criteria were kept in mind, the most important of which was the 
effect of individual differences among the 29 teachers involved. Since 
it was impractical, if not impossible, to select teachers with like 
personalities, a special attempt was made to select a review procedure 
which was practical, from an economic and technological standpoint, and 
which would minimize the effect of the teacher during the review. Sev- 
eral review procedures were explored including overhead transparencies, 
charts, movie films, filmstrips, recordings on both discs and tapes, 
scripts to be read by the teachers, and various combinations of these. 

The production expense involved in some of these procedures was prohibi- 
tive, and in other cases, the necessary record playing and projection 
equipment was not available in the schools. 

After considering the various alternative procedures, the film- 
strip was selected as the best review procedure. Filmstrip projectors 
were found in all participating schools and the cost of a filmstrip was 
much less than that of some of the other audio-visuals considered. In 
essence, the review procedure consisted of the teacher showing his stu- 
dents a series of filmstrip frames on which were contained information 
the students had earlier learned from the programed textbook. Designated 
portions of the script contained ii the frames were read by the teacher, 
others by his students. 

Step 3; Constructing the Review Filmstrip 
In constructing the review filmstrip, care was taken to include 
only the factual information covered in the programed unit. Each important 
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concept in the program was identified and filmstrip frames were constructed 
which would provide a review of each concept. In some cases only a single 
filmstrip frame was necessary for reviewing a concept; in other cases, two 
or more frames were deemed necessary. The same ordering of concepts used 
in the programed unit was used in the filmstrip. 

The frames for the review filmstrip were constructed on five- inch 
by eight- inch index cards. All wording on the cards was typed. A repro- 
duction of the cards from which the filmstrip was photographed is in 
Appendix A. 

The completed cards were sent to George W. Colburn Laboratories, 
Inc., l64 North Wacker Drive, Chicago 6, Illinois, the firm which actually 
photographed the filmstrip and which duplicated it in sufficient quantity 
for use in the experiment. 

Step 4; Developing the Criterion Test 
A criterion test was developed which would serve two purposes, 
namely, to measure how much the students learned at the end of the treat- 
ment and how much they retained thirty days later. The first step in 
developing the criterion test was to determine the educational objectives 
of the program. These objectives were not provided in the programed unit; 
therefore, it was necessary to determine them, by inference, from the text 
of the program itself. The objectives are listed below; 

Educational Objectives of the Programed Unit 

1. Mien presented with a random list of the eight steps in the 
order of business, the student should be able to arrange them in their 
proper order. 
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2. When tresented with an item of business, the student should 
be able to place it vdthin the proper step in the order of business. 

3 . When presented with the steps for making and disposing of a 
motion, the student should be able to arrange them in their correct order. 

4. When presented with a parliamentary situation, the student 
should be able to recognize the correct procedure for handling the situa- 
tion. 

5* The student should be able to recognize a properly worded 

motion. 

6. When presented with a properly worded motion, the student 
should be able to amend it by each of the three common methods. 

7* When presented with a situation in which an amendment to a 
motion has been amended, the student should know the correct sequence 
for disposing of the motion and the amendments. 

8. The student should be able to recall the four general class- 
ifications of motions. 



10. The student should be able to select the proper motion to 
take care of a given situation whicn may arise during a meeting. 

11. When presented with a random list of all privileged and 
subsidiary motions, the student should be able to arrange them in their 
proper order of precedence. 

The criterion test used in this experiment was original and un- 
refined. Although an attempt was made to construct an instrument which 
would possess validity, there is no assurance, beyond the judgment of 
the writer, that it measured what it was purported to measure. 



9 . The student should be able to classify any given motion. 




s 
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In an attempt to establish validity in the criterion test, two 
types of validity were stressed, content validity and face validity. A 
test is said to have content validity to the extent that each objective 
of the unit is emphasized in the test to the same degree that it is 
emphasized in the unit. A test is said to have face validity to the 
extent that the test questions appear to measure the degree to which 
the objectives have been met. 

In order to attain content validity, the number of frames in each 
subtopic of the program was counted and this count was used to determine 
the weight to be given to each subtopic, or the extent to which each sub- 
topic was to be emphasized in the test. The first subtopic contained 
55 frames, the second contained 63 j the third 52, and the fourth 4o4 
frames. Each subtopic was given a weight equal to the percent the number 
of frames on that subtopic was of the total frames of the program. In 
this manner the foiu* subtopics of the program were given the weights 
nine, fourteen, nine, and sixty-eight respectively. Similarly, each sub- 
topic was divided among its objectives accordingly. The table below 
shows the emphasis given to each objective as determined by the above 
procediire . 

Using the table below as a guide, the test question developed to 
evaluate a particular objective was given the same emphasis as the objec- 
tive. For example, the test question for objective three was given a 
weight of four percent of the total test score of one hundred. 

Face validity was striven for by selecting the type of test ques- 
tion which, in the judgment of the writer would best evaluate a particular 
objective. Since the objectives were stated rather explicitly in terms 
of the terminal behavior it was expected the students should display when 
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Table 1 . Percent of Total Emphasis Given to Each Subtopic 
and Each Objective in the Programed Unit. 



Subtopic 

Number 


Percent of 
Total Emphasis 


Objective 

Number 


Percent of 
Total Emphasis 


1 


cr\ 


1 


3% 


( 55 frames) 




2 


6& 


2 


lk% 


3 


k% 






4 




( 83 frames) 




5 


hi 


3 




6 


yjo 


( 52 frames) 




7 




k 


68% 


8 


h% 






9 


2h% 






10 


30% 


(4o4 frames) 




11 


10% 



they finished the program., test questions were constructed which called 
upon the student to display this terminal behavior. For example, objec- 
tive number one was: VJhen presented with a random list of the eight 

steps in the order of business, the student should be able to arrange 
them in their proper order. The test question constructed to evaluate 
the degree to which this objective was accomplished was test question 
number one which read: Indicate the correct order of the above steps 

by filling in the blanks below with the appropriate letter. Above the 
question were listed in random order, the eight steps in the order of 
business, each preceeded by a small letter; below the question the stu- 
dents were provided with eight blanks labeled Step 1 through Step 8. 

The placing of the correct small letters in the proper blanks was the 
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terminal behavior called for, within the limitations of a paper-pencil 
test. 

An inherent element in this esqperiment is the basic assumption 
that the procedures used by the writer in devising the evaluative instru- 
ment (paper-pencil test) resulted in a valid instrument. A copy of the 
test is in Appendix B. 

Step 5: Selecting the Sample 

Ninth grade students enrolled in vocational agriculture in New 
York were selected as the subjects of this experiment. Parliamentary 
procedure is usually taught at the ninth grade level to students of 
vocational agriculture as part of the leadership training activities of 
the Future Farmers of America Organization. 

All vocational agriculture departments operated by boards of 
education in New York were given a different niiniber. Then, enough schools 
were selected at random, using a table of random numbers, until approxi- 
mately 350 students were secured as subjects. Some of the schools selected 
were unable to participate. The final sample involved 29 teachers in 28 
schools . 

Some of the teachers reported that whereas their first year agri- 
culture class was composed of predominantly ninth graders, there were 
some tenth, eleventh and twelvth grade students included also. While 
all students within the classes selected were allowed to participate, 
only data from ninth and tenth grade students were used in the analysis. 

The total number of students from whom complete data were collected was 
279 j of which 228 were ninth graders and 51 were tenth graders. The use 
of these few tenth grade students was justified by using reading ability 



o 
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as a statistical control in the analysis. 

Step 6 : Obtaining Scores on the Covariable 

All students were tested for reading ability using Form A of the 
Nelson- Denny Reading Test. This test yields a score based on the indi- 
vidual's vocabulary and reading comprehension ability. A third part of 
the test, reading rate, was considered of no importance because the 
students were to be allowed to proceed through the program at their own 
pace. This test was administered in the standard manner by the cooperat- 
ing teachers well in advance of the actual starting date for the treat- 
ment. A copy of the instructions for administering the reading test is 
in Appendix C. 

Step 7: Assigning Classes to Treatment Groups 

The twenty- nine classes were ranked from one to twenty- nine on 
mean reading score. Since class size varied widely, the treatment groups 
were adjusted for number as the classes were assigned to them. For ex- 
ample, the five highest classes on mean reading score contained seven, 
seventeen, five, ten, and twelve students, respectively. For the p\npose 
of assignment to treatment groups, the class containing five students was 
combined with the class of twelve, and the class of seven students was 
combined with the class of ten. This resulted in three groups of seven- 
teen students each. The groups were then assigned at random to one of 
the three treatment groups. Such a procedure was followed until all 
classes were assigned to a treatment. 

This method of assignment to treatment groups met the requirements 
for random assignment and also insured that the three groups were composed 
of students of approximately equal reading ability. 
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Step 8: Administering the Treatments 

The teachers were contacted by letter relative to the date for 
starting the treatment. Four starting dates were proposed and each teacher 
was asked to indicate his choice on a return postcard. The four dates were 
January 9th, l6th, 23rd, and 30th, of 196? . This selection allowed the 
teacher to plan ahead for the puip>oses of both finishing any pending in- 
structional units and also to select a time during which there would be 

the least number of interruptions. 

In administering the treatments, the group having spaced review 
con 5 >leted one segment of the program, then had a review of that segment 
by viewing the corresponding segment of the filmstrip. This pai^tern was 
continued until the instructional period was over. The group ha\'lng 
blocked review completed the entire program, then viewed the entire 
filmstrip. The group having program only did not view the filmstrip. 

The instructional materials were mailed to the teachers along 
with explicit instructions for administering the appropriatt treatment 
to their students. Copies of the letters containing these instructions 
are in Appendix D. All instructional materials were donated to the 
participating schools upon termination of the experlinent . 

step 9: Admini stering the Criterion Test 

The test for learning was administered by the teacher during the 
first whole class period after the treatment was completed. The re-test 
for retention was administered thirty days after the first test. In 
cases where the thirtieth day fell on a weekend, the retention test was 
given on the preceeding Friday or the following Monday. The test time 
was standardized at forty minutes. 
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Step 10; Analyzing the Data 

Scores on the three variables of interest, namely, reading ability, 

amount of learning, and amount of retention, were punched on IBM cards 

so that the data analysis could be facilitated by using a Control Data 

l604 Computer housed in the Cornell University Computing Center in Ithaca o 

A program called "OKVAB" was available at the Center which was designed 

to handle a one-way analysis of covavriance and a multiple t-test of means, 

as adjusted by covariance. The mathematics of these procedures may be 

20 

found in Federer. 

Two one-way analyses of covariance were run, the first on learning 
test scores adjusted for variations in reading score, and the second on 
retention test scores adjusted for variations in reading score. The co- 
variance analyses provided the writer vriLth F values whiich were con^ared 
with tabled va.lues of F to determine significance. The F value is com- 
puted by di^n.ding the mean square (or the variance) between groups by the 
mean square (or the variance) within groups. Since it was hypothesized 
that there would be greater variation between groups than within groups, 
the F value should be greater than one. If the F value is as large or 
larger then the tabled value of F for a predetermii ed level of proba- 
bility, then a significant difference exists between the groups. This 
difference would be attributable to treatment effects. 

A multiple t-test of adjusted mean test scores of the three treat- 
ment groups "was run s3jnijLlt?,neously with the analysis of covariance. The 
multiple t-test of adjusted means yielded an LSD (Least Significant Differ- 
ence) value which takes the place of the tabled values of t used in the 

20. Walter T. Feclsrer, E^tperiment al Design (New York: The Macmillan 

Company, 1955 )> pp"* 20-21 and"pT*SS^ 
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more common t-test. If the difference between any two adjusted means 
is equal to or larger than the LSD value confuted for that pair of means, 
then a significant difference exists between the adjusted means. 

Throughout the experiment the .05 level of probability was used 
to determine if significant differences existed. VJhen it is said that 
differences are significant at the .05 level of probability, that means 
the differences could have happened by chance no more than five times 



in 100. 



CHAPTER III 



FINDINGS 

The primary objective of this experiment was to determine the 
relative effectiveness of supplementing programed instruction with 
blocked versus spaced review. Four deductions were drawn from the 
theoretical framework and posed as the hypotheses around which the ex- 
periment was designed. These are reproduced here for the purpose of 
convenient reference. 

hypothesis 1: Students who have program texts supplemented with 

review will learn more than those who have only programed texts, as 
measured by a test for learning administered immediately after the 
instructional period. 

Hj^othesis 2: Students who have programed texts supplemented 

with spaced review will learn more than those who have programed texts 
supplemented with blocked review, as measured by a test for learning 
administered immediately after the instructional period. 

Hypothesis 3: Students who have programed texts supplemented 

with review will retain more than those who have only programed texts, 
as measured by a test for retention administered after an interval of 
30 days. 

I^ypothesis 4: Students who have programed texts supplemented 

with spaced review will retain more than those who have programed texts 
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supplemented with "blocked review, as measured by a test for retention 
administered after an interval of 30 days. 
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Since the two variables of interest were learning and retention, 
it was convenient to structure this chapter in terms of these two vari- 
ables. 

Learning : The first analysis done was an analysis of covariance on learn- 

ing test scores adjusted for variations in reading score. This analysis 
resulted in a significant F value at the .05 level of probability, meaning 
that there was greater variation between groups than within groups. This 
difference was attributed to treatment effects. The results of this 
analysis f.re presented below, in Table 2. 

Table 2. Analysis of Covariance of Scores 
on Test for Learning. 



Sources of 
Variation 


Degrees of 
Freedom 


Sums of 
Squares 


Mean 

Squares 


Between 

Groups 


2 


1026.50 


513.25 


Within 

Groups 


275 


4o4l6.oo 


146.97 


Total 


277 


4i442.50 





P.05 == 3.04 (with 2 and 275 degrees of freedom) 



F = = 3.U5 



ll(6.97 



Although the above analysis of covariance yielded a significant 
P value, it did not show between which of the three groups was the 
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significant difference attributable. For this, a multiple t-test was run 
between all possible pairs of the three adjusted group means in order to 
locate the one or more pairs of means which were significantly different 
from each other. Table 3 shows the results of this multiple t-test. 



Table 3 , Multiple t-test for Differences Between Adjusted 
Group Means of Learning Test Scores, Adjusted 
for Variations in Reading Score, 



Group 


Adjusted 

Means 


Differences 

Between 

Means 


Least 

Significant 

Difference^*" 


1 (Program plus blocked 
review, n = 99) 


52.51 


X^-X2=4.66 


4,o4 


2 (Program plus spaced 
review, n = 88) 


47.85 


X^-X2=1.96 


4,20 


3 (Program only, n = 92) 


49.81 




4,05 


*At the ,05 level of probability 



From Table 3 it can be seen that there was no significant differ- 
ence between the means of Group 1 and Group 3 and between those of Group 2 
and Group 3j a.s revealed by the fact that the differences between the 
adjusted means in question were less than the values required at the 
,05 level of probalbility, A significant difference did exist, however, 
between Group 1 and Group 2 with the adjusted mean for Group 1, 52,51> 
being significantly higher than that for Group 2, 47,85, This is indicated 
by the difference of 4,66 being greater than the Least Significant Differ- 
ence of 4,04, 
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On the basis of this first analysis Hypothesis 1, stating that 
the groups having review will learn more than the group having only the 
program, was not accepted because neither of the groups which had the 
program supplemented with review did better than the group which had only 
the program. Hypothesis 2, stating that the group having spaced review 
will learn more than the group having blocked review, was rejected because 
the finding was opposite to that hypothesized; that is, the group having 
blocked review did significantly better than the group having spaced 
review. 

Retention : A second analysis of covariance was run on retention test scores 

adjusted for variations in reading score. This analysis yielded a non- 
significant F value at the .05 level indicating no significant difference 
in performance between any two of the three groups. This analysis is 
presented in Table 4. The adjusted mean scores of the three groups on 
the retention test were: Group 1 - 49.1, Group 2 - 47.3a Group 3 - 49.9* 



Table 4 . Analysis of Covariance of Scores 
on Test for Retention. 



Sources of 
Variation 


Degrees of 
Freedom 


Sums of 
Squares 


Mean 

Squares 


Between 

Groups 


2 


308.22 


154.11 


Within 

Groups 


275 


39564.00 


143.87 


Total 


277 


39872.22 





P.05 = 3.04 (with 2 and 275 degrees of freedom) 



F 



= 154.11 ^ 
143 .^ 7 “ 



1.07 



The fact that the F value in this analysis ms just more than 1 
means that there was almost as much vau^iation within groups as between 
groups, and any treatment effect which was present during the test for 
learning was no longer present 30 days later, during the test for reten- 
tion. On the basis of this second analysis Hypothesis 3? stating that 
students having review will retain more than those having program only, 
and Hypothesis 4, stating that students having spaced review will retain 
more than those having blocked review, were not accepted. 



CHAPTER IV 



CONCLUSIONS, RECOMMENDATIONS, AND SUMMARY 

The findings of this study reported in the preceding chapter were 
discussed in terms of learning, retention, and the four hypotheses posed. 
It remains for the writer to synthesize from these findings concise euid 
logical conclusions. It should be remenibered, however, that the conclu- 
sions are drawn only within the realm of the findings of this e:q)eriment 
and should be interpreted only within that framework. Included also in 
this chapter are recommendations and a summary. 

Conclusions 

Conclusion 1 ; Supplementing a program with either spaced or 
blocked review has no beneficial effect in either learning or retention, 
over administering the program alone. 

Conclusion 2 ; Supplementing a progriun with blocked review will 
result in more learning than supplementing it with spaced review although 
it does not produce a beneficial effect in restention. 

Since the writer is unaware of any other study which this experi- 
ment replicates, the conclusions cannot be compared with others. A com- 
parison can be made, however, with the conclusion drawn by Reynolds in 
his series of experiments with spaced review, as reported in Chapter I. 
Reynolds concluded that "Spacing of review sequences between interpolated 
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2’L 

learning material facilitated retention." ' The ciirrent experiment failed 
to demonstrate this; however, it should he remembered that this e:q)eriment 
employed both a different definition and means of review. 

Re commc ndat ions 

Two types of recommendations were considered appropriate for inclu- 
sion in the final chapter of this thesis. These are recommendations for 
additional research and recommendations for supplementing programed in- 
struction with review. Because of the importance of the problem which 
resulted in the construction of the current experiment and because of the 
dearth of studies dealing with spaced and blocked review as a supplement 
to programed instruction, the writer recommends that additional experi- 
ments be conducted in this area. This research may take the form of 
replications of the current experiment with variations of such faotors 
as subject matter of the program, method of review, grade level of the 
sample, the covariable, or others, according to the judgment of the 
researchers. 

The writer* s recommendation regarding the use of review as a 
supplement to programed ihstruction was influenced by the soundness of 
the theoretical framework used on the,* basis for the current ejqperiment 
and by the lack of other evidence supporting the negative findings of 
the current study. On these bases the writer recommends that in a situa- 
tion where the teacher finds it necessary to supplement a program with 
review, he employ spaced review. 



\ 




21. Reynolds and Glaser, p. 39* 
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Summary 

This study involved an experiment , the purpose of which was to 
determine the relative effectiveness of supplementing programed instruc- 
tion with hlocked versus spaced re-'d.eWo It was hypothesized that: (l) 

students who have program texts supplemented with review will learn more 
than those who have only programed texts, as measured by a test for 
learning administered immediately after the instructional period; (2) 
students who ha.ve programed texts supplemented with spaced review will 
learn more than those who have programed texts supplemented with blocked 
review, as measured by a test for learning administered immediately after 
the instructional period; (3) students who have programed texts supple- 
mented with review will retain more than those who have only programed 
texts, as measured by a test for retention administered after an interval 
of 30 days; and (4) students who have programed texts supplemented with 
spaced review will retain more than those who have programed texts supple- 
mented with blocked review, as measured by a test for retention administered 
after an interval of 30 days. 

Students enrolled in first-year vocational agriculture in twenty- 
seven New York Schools, selected random, m?.de up 'ohe sample. These 
students were first tested for reading ability, then intact classes 
were randomly assigned to one of three treatment groups using a procedure 
which insured groups of approxim.ately equal mean reading ability. These 
treatment groups were: Group 1 - program plus blocked review; Group 2 - 

program plus spaced revi.ew; end Group 3 - program only. Review was pro- 
vided by a filmstrip ma,de especirbly for use in the experiment. The group 
having spaced review conroleted Oiie segment of the program, then had a review 
of that segment by viewing the corresponding segment of the filmstrip. This 
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pattern was continued until the instructional period was over. The group 
having blocked review completed the entire program, then viewed the entire 
review filmstrip. The group having program only did not view the film- 
strip. 

At the end of the instructional period, all students were tested 
for learning using an objective type paper-pencil test constructed by the 
writer. They were re-tested for retention after a thirty-day interval. 
CoQ^lete sets of data, which were made up of a reading test score, a 
learning test score, and a retention test score, were collected on 279 
students of which 228 were ninth graders and 51 were tenth graders. 

The data were analyzed by two one-way analyses of covariance, the 
first being run on learning test scores adjusted for variations in read- 
ing score, and the second on retention test scores adjusted for variations 
in reading score. Neither spaced review nor blocked review resulted in 
significantly better performance than program only as measured by the 
learning test or by the retention test. Blocked review was significantly 
better than spaced review as measured by the learning test, but not by 



the retention test 
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APPKiDIX 



(The appendices of this thesis have been deleted for the 
purposes of this final report. If it is desired to view the 
appendix material, the reader is referred to the complete 
thesis a*vedlable in the Cornell University Library. ) 



